Multiagent Q-Learning: Preliminary Study on Dominance between the Nash and Stackelberg Equilibriums
نویسندگان
چکیده
Some game theory approaches to solve multiagent reinforcement learning in self play, i.e. when agents use the same algorithm for choosing action, employ equilibriums, such as the Nash equilibrium, to compute the policies of the agents. These approaches have been applied only on simple examples. In this paper, we present an extended version of Nash Q-Learning using the Stackelberg equilibrium to address a wider range of games than with the Nash Q-Learning. We show that mixing the Nash and Stackelberg equilibriums can lead to better rewards not only in static games but also in stochastic games. Moreover, we apply the algorithm to a real world example, the automated vehicle coordination problem.
منابع مشابه
Analyzing Multi-agent Reinforcement Learning Using Evolutionary Dynamics
In this paper, we show how the dynamics of Q-learning can be visualized and analyzed from a perspective of Evolutionary Dynamics (ED). More specifically, we show how ED can be used as a model for Qlearning in stochastic games. Analysis of the evolutionary stable strategies and attractors of the derived ED from the Reinforcement Learning (RL) application then predict the desired parameters for R...
متن کاملApprentissage de la coordination multiagent. Une méthode basée sur le Q-learning par jeu adaptatif
Current algorithmes on multiagent learning are for almost limited since they cannot manage the multiplicity of Nash equilibria and thus converge to the Pareto-optimal. To alleviate this, we propose here a learning mechanism extending the Q-learning to non-cooperative stochastique games. This learning mechanism converges to Pareto-optimal equilibria in selfplay. We present experimental results s...
متن کاملMultiagent Reinforcement Learning: Theoretical Framework and an Algorithm
In this paper we adopt general sum stochas tic games as a framework for multiagent re inforcement learning Our work extends pre vious work by Littman on zero sum stochas tic games to a broader framework We de sign a multiagent Q learning method under this framework and prove that it converges to a Nash equilibrium under speci ed condi tions This algorithm is useful for nding the optimal strateg...
متن کاملThe Dynamics of Reinforcement Learning in Cooperative Multiagent Systems
Reinforcement learning can provide a robust and natural means for agents to learn how to coordinate their action choices in multiagent systems. We examine some of the factors that can influence the dynamics of the learning process in such a setting. We first distinguish reinforcement learners that are unaware of (or ignore) the presence of other agents from those that explicitly attempt to lear...
متن کاملPricing and Advertising Decisions in a Three-level Supply Chain with Nash, Stackelberg and Cooperative Games
Pricing and advertising are two important marketing strategies in the supply chain management which lead to customer demand’s increase and therefore higher profit for members of supply chains. This paper considers advertising, and pricing decisions simultaneously for a three-level supply chain with one supplier, one manufacturer and one retailer. The amount of market demand is influenced ...
متن کامل